Progress in deep learning has spawned great successes in many engineering applications. As a prime example, convolutional neural networks, a type of feedforward neural networks, are now approaching -- and sometimes even surpassing -- human accuracy on a variety of visual recognition tasks. Here, however, we show that these neural networks and their recent extensions struggle in recognition tasks where co-dependent visual features must be detected over long spatial ranges. We introduce a visual challenge, Pathfinder, and describe a novel recurrent neural network architecture called the horizontal gated recurrent unit (hGRU) to learn intrinsic horizontal connections -- both within and across feature columns. We demonstrate that a single hGRU layer matches or outperforms all tested feedforward hierarchical baselines including state-of-the-art architectures with orders of magnitude more parameters.

This task – the Pathfinder Challenge – has been used in the neuroscience literature and requires deciding whether two dots in an image are connected by a path made of short line segments. They show that a single layer of hGRU can solve this task almost perfectly, while CNNs need to be quite deep to achieve comparable performance and require orders of magnitude more parameters. Strengths: The paper is well motivated and conceptually very clear. The Pathfinder challenge uses simple images to generate a non-trivial and interesting task. The paper shows a limitation of CNNs and proposes an effective solution using a gated recurrent model. The result that a one-layer recurrent model can solve the task is quite remarkable. Ablation studies and comparisons with other models show that the proposed hGRU model maximizes the ratio of performance to the number of parameters. Weaknesses: - The hGRU architecture seems pretty ad-hoc and not very well motivated.

Progress in deep learning has spawned great successes in many engineering applications. As a prime example, convolutional neural networks, a type of feedforward neural networks, are now approaching -- and sometimes even surpassing -- human accuracy on a variety of visual recognition tasks. Here, however, we show that these neural networks and their recent extensions struggle in recognition tasks where co-dependent visual features must be detected over long spatial ranges. We introduce a visual challenge, Pathfinder, and describe a novel recurrent neural network architecture called the horizontal gated recurrent unit (hGRU) to learn intrinsic horizontal connections -- both within and across feature columns. We demonstrate that a single hGRU layer matches or outperforms all tested feedforward hierarchical baselines including state-of-the-art architectures with orders of magnitude more parameters.